Application of Bayesian approaches in drug development: starting a virtuous cycle.

The pharmaceutical industry and its global regulators have routinely used frequentist statistical methods, such as null hypothesis significance testing and p values, for evaluation and approval of new treatments. The clinical drug development process, however, with its accumulation of data over time, can be well suited for the use of Bayesian statistical approaches that explicitly incorporate existing data into clinical trial design, analysis and decision-making. Such approaches, if used appropriately, have the potential to substantially reduce the time and cost of bringing innovative medicines to patients, as well as to reduce the exposure of patients in clinical trials to ineffective or unsafe treatment regimens. Nevertheless, despite advances in Bayesian methodology, the availability of the necessary computational power and growing amounts of relevant existing data that could be used, Bayesian methods remain underused in the clinical development and regulatory review of new therapies. Here, we highlight the value of Bayesian methods in drug development, discuss barriers to their application and recommend approaches to address them. Our aim is to engage stakeholders in the process of considering when the use of existing data is appropriate and how Bayesian methods can be implemented more routinely as an effective tool for doing so.

Accurate Collection of Reasons for Treatment Discontinuation to Better Define Estimands in Clinical Trials.

BACKGROUND: Reasons for treatment discontinuation are important not only to understand the benefit and risk profile of experimental treatments, but also to help choose appropriate strategies to handle intercurrent events in defining estimands. The current case report form (CRF) commonly in use mixes the underlying reasons for treatment discontinuation and who makes the decision for treatment discontinuation, often resulting in an inaccurate collection of reasons for treatment discontinuation. METHODS AND RESULTS: We systematically reviewed and analyzed treatment discontinuation data from nine phase 2 and phase 3 studies for insulin peglispro. A total of 857 participants with treatment discontinuation were included in the analysis. Our review suggested that, due to the vague multiple-choice options for treatment discontinuation present in the CRF, different reasons were sometimes recorded for the same underlying reason for treatment discontinuation. Based on our review and analysis, we suggest an intermediate solution and a more systematic way to improve the current CRF for treatment discontinuations. CONCLUSION: This research provides insight and directions on how to optimize the CRF for recording treatment discontinuation. Further work needs to be done to build the learning into Clinical Data Interchange Standards Consortium standards. CLINICAL TRIALS: Clinicaltrials.gov numbers: NCT01027871 (Phase 2 for type 2 diabetes), NCT01049412 (Phase 2 for type 1 diabetes), NCT01481779 (IMAGINE 1 Study), NCT01435616 (IMAGINE 2 Study), NCT01454284 (IMAGINE 3 Study), NCT01468987 (IMAGINE 4 Study), NCT01582451 (IMAGINE 5 Study), NCT01790438 (IMAGINE 6 Study), NCT01792284 (IMAGINE 7 Study).

Estimating the treatment effect for adherers using multiple imputation.

Randomized controlled trials are considered the gold standard to evaluate the treatment effect (estimand) for efficacy and safety. According to the recent International Council on Harmonization (ICH)-E9 addendum (R1), intercurrent events (ICEs) need to be considered when defining an estimand, and principal stratum is one of the five strategies to handle ICEs. Qu et al. (2020, Statistics in Biopharmaceutical Research 12:1-18) proposed estimators for the adherer average causal effect (AdACE) for estimating the treatment difference for those who adhere to one or both treatments based on the causal-inference framework, and demonstrated the consistency of those estimators; however, this method requires complex custom programming related to high-dimensional numeric integrations. In this article, we implemented the AdACE estimators using multiple imputation (MI) and constructed confidence intervals (CIs) through bootstrapping. A simulation study showed that the MI-based estimators provided consistent estimators with the nominal coverage probabilities of CIs for the treatment difference for the adherent populations of interest. As an illustrative example, the new method was applied to data from a real clinical trial comparing two types of basal insulin for patients with type 1 diabetes.

Assessing and communicating heterogeneity of treatment effects for patient subpopulations: The hardest problem there is.

Heterogeneity is an enormously complex problem because there are so many dimensions and variables that can be considered when assessing which ones may influence an efficacy or safety outcome for an individual patient. This is difficult in randomized controlled trials and even more so in observational settings. An alternative approach is presented in which the individual patient becomes the "subgroup," and similar patients are identified in the clinical trial database or electronic medical record that can be used to predict how that individual patient may respond to treatment.

Implementation of tripartite estimands using adherence causal estimators under the causal inference framework.

Intercurrent events (ICEs) and missing values are inevitable in clinical trials of any size and duration, making it difficult to assess the treatment effect for all patients in randomized clinical trials. Defining the appropriate estimand that is relevant to the clinical research question is the first step in analyzing data. The tripartite estimands, which evaluate the treatment differences in the proportion of patients with ICEs due to adverse events, the proportion of patients with ICEs due to lack of efficacy, and the primary efficacy outcome for those who can adhere to study treatment under the causal inference framework, are of interest to many stakeholders in understanding the totality of treatment effects. In this manuscript, we discuss the details of how to estimate tripartite estimands based on a causal inference framework and how to interpret tripartite estimates through a phase 3 clinical study evaluating a basal insulin treatment for patients with type 1 diabetes.

Détente: A Practical Understanding of P values and Bayesian Posterior Probabilities.

Null hypothesis significance testing (NHST) with its benchmark P value < 0.05 has long been a stalwart of scientific reporting and such statistically significant findings have been used to imply scientifically or clinically significant findings. Challenges to this approach have arisen over the past 6 decades, but they have largely been unheeded. There is a growing movement for using Bayesian statistical inference to quantify the probability that a scientific finding is credible. There have been differences of opinion between the frequentist (i.e., NHST) and Bayesian schools of inference, and warnings about the use or misuse of P values have come from both schools of thought spanning many decades. Controversies in this arena have been heightened by the American Statistical Association statement on P values and the further denouncement of the term "statistical significance" by others. My experience has been that many scientists, including many statisticians, do not have a sound conceptual grasp of the fundamental differences in these approaches, thereby creating even greater confusion and acrimony. If we let A represent the observed data, and B represent the hypothesis of interest, then the fundamental distinction between these two approaches can be described as the frequentist approach using the conditional probability pr(A | B) (i.e., the P value), and the Bayesian approach using pr(B | A) (the posterior probability). This paper will further explain the fundamental differences in NHST and Bayesian approaches and demonstrate how they can co-exist harmoniously to guide clinical trial design and inference.

Efficiency of two sample tests via the restricted mean survival time for analyzing event time observations.

In comparing two treatments with the event time observations, the hazard ratio (HR) estimate is routinely used to quantify the treatment difference. However, this model dependent estimate may be difficult to interpret clinically especially when the proportional hazards (PH) assumption is violated. An alternative estimation procedure for treatment efficacy based on the restricted means survival time or t-year mean survival time (t-MST) has been discussed extensively in the statistical and clinical literature. On the other hand, a statistical test via the HR or its asymptotically equivalent counterpart, the logrank test, is asymptotically distribution-free. In this article, we assess the relative efficiency of the hazard ratio and t-MST tests with respect to the statistical power under various PH and non-PH models theoretically and empirically. When the PH assumption is valid, the t-MST test performs almost as well as the HR test. For non-PH models, the t-MST test can substantially outperform its HR counterpart. On the other hand, the HR test can be powerful when the true difference of two survival functions is quite large at end but not the beginning of the study. Unfortunately, for this case, the HR estimate may not have a simple clinical interpretation for the treatment effect due to the violation of the PH assumption.

Considerations for Evaluating Treatment Effects From Randomized Clinical Trials.

This article focuses on the choice of treatment effect measures in randomized clinical trials (RCTs). Traditionally, an intention-to-treat (ITT) analysis is conducted with an implicit understanding that a treatment-policy effect is of greatest interest. In this article we contend that this approach may not always provide accurate information about clinically meaningful treatment effects, and we present an argument that for any RCT it is desirable to require an explicit definition of what treatment effect is of primary interest, known as the "estimand." We will discuss the limitations of the traditional ITT effect measures as well as the state-of-the art thinking with regard to estimands. Furthermore, we will offer alternate choices that acknowledge that treatments have multiple effects.

Making what's advanced today routine tomorrow.

Statistical principles and ongoing proliferation of novel statistical methodologies have dramatically improved the clinical drug development process. This journey over the last seven decades reshaped the pharmaceutical industry and regulatory agencies, highlighted the importance of statistical thinking in drug development and decision-making, and, most importantly, improved the lives of countless patients around the world. Some significant highlights in the history of this journey are recounted here as well as some exciting opportunities of what the future may hold for the science and profession of statistics.

Identifying potential adverse events dose-response relationships via Bayesian indirect and mixed treatment comparison models.

Patients and prescribers need to be fully informed regarding the safety profile of approved medications. This includes knowledge and information regarding whether an adverse event of interest exhibits a potential dose-response relationship. In order to thoroughly evaluate whether an adverse event rate increases with increasing dose level, evidence from multiple clinical trials needs to be combined and analyzed. The various clinical trials that need to be combined often include different dose levels. If one evaluates the dose-response relationship by including only the trials with all of the common dose levels, this will lead to the exclusion of potentially several clinical trials as well as dose levels, and thus the loss of important information. Other methods, such as crudely pooling patients on the same dose level across different studies, are subject to bias due to the breakdown of randomization. It is preferable to include all studies and relevant dose levels, even if all studies do not contain the same dose levels. Bayesian methodology has been shown to be useful in the context of indirect and mixed treatment comparison methods, to combine information from different therapies in different studies in order to make treatment effect inferences. This type of approach is foundational to the models presented here, but instead of modeling different dose arms in different studies, we extend the methodology to allow for assessment of the dose-response relationship across multiple clinical trials. In this paper, we propose three Bayesian indirect/mixed treatment comparison models to assess adverse event dose-response relationships. These three models are designed to handle binary responses and time to event responses. We apply the methods to real data sets and demonstrate that our proposed methods are useful in discovering potential dose-response relationships.

Subgroup identification from randomized clinical trial data.

We consider the problem of identifying a subgroup of patients who may have an enhanced treatment effect in a randomized clinical trial, and it is desirable that the subgroup be defined by a limited number of covariates. For this problem, the development of a standard, pre-determined strategy may help to avoid the well-known dangers of subgroup analysis. We present a method developed to find subgroups of enhanced treatment effect. This method, referred to as 'Virtual Twins', involves predicting response probabilities for treatment and control 'twins' for each subject. The difference in these probabilities is then used as the outcome in a classification or regression tree, which can potentially include any set of the covariates. We define a measure Q(Â) to be the difference between the treatment effect in estimated subgroup  and the marginal treatment effect. We present several methods developed to obtain an estimate of Q(Â), including estimation of Q(Â) using estimated probabilities in the original data, using estimated probabilities in newly simulated data, two cross-validation-based approaches, and a bootstrap-based bias-corrected approach. Results of a simulation study indicate that the Virtual Twins method noticeably outperforms logistic regression with forward selection when a true subgroup of enhanced treatment effect exists. Generally, large sample sizes or strong enhanced treatment effects are needed for subgroup estimation. As an illustration, we apply the proposed methods to data from a randomized clinical trial.

Early improvement in pain predicts pain response at endpoint in patients with fibromyalgia.

UNLABELLED: An unanswered, but clinically important question is whether there are early indicators that a patient might respond to duloxetine treatment for fibromyalgia pain. To address this question, pooled data from 4 double-blind, placebo-controlled trials in duloxetine-treated patients (N = 797) with primary fibromyalgia as defined by the American College for Rheumatology were analyzed. Classification and Regression Tree (CART) analysis was used to determine what level of early pain improvement as measured by the 24-hour average pain severity question on the Brief Pain Inventory (BPI) best predicted later response. The predictor variables tested were 10, 15, 20, 25, and 30% decrease in BPI 24-hour average pain from baseline to Week 1 and Week 2. The results of the CART analysis showed that for patients with ≥15% improvement in pain at Week 1 and ≥30% improvement at Week 2, the probability of response at 3 months was 75%. For patients with <15% improvement at both Week 1 and Week 2, the probability of not responding at 3 months was 86%. Quantifiable early improvement in pain during the first 2 weeks of treatment with duloxetine was highly predictive of response or nonresponse after 3 months of treatment. PERSPECTIVE: This article presents early indicators that can highly predict later pain response or nonresponse in fibromyalgia patients treated with duloxetine. The results may aid clinicians to predict the likelihood of response at 3 months within the first 2 weeks of treatment.

Identification of early changes in specific symptoms that predict longer-term response to atypical antipsychotics in the treatment of patients with schizophrenia.

BACKGROUND: To identify a simple decision tree using early symptom change to predict response to atypical antipsychotic therapy in patients with (Diagnostic and Statistical Manual, Fourth Edition, Text Revised) chronic schizophrenia. METHODS: Data were pooled from moderately to severely ill patients (n = 1494) from 6 randomized, double-blind trials (N = 2543). Response was defined as a ≥ 30% reduction in Positive and Negative Syndrome Scale (PANSS) Total score by Week 8 of treatment. Analyzed predictors were change in individual PANSS items at Weeks 1 and 2. A decision tree was constructed using classification and regression tree (CART) analysis to identify predictors that most effectively differentiated responders from non-responders. RESULTS: A 2-branch, 6-item decision tree was created, producing 3 distinct groups. First branch criterion was a 2-point score decrease in at least 2 of 5 PANSS positive items (Week 2). Second branch criterion was a 2-point score decrease in the PANSS excitement item (Week 2). "Likely responders" met the first branch criteria; "likely non-responders" did not meet first or second criterion; "not predictable" patients did not meet the first but did meet the second criterion. Using this approach, response to treatment could be predicted in most patients (92%) with high positive predictive value (79%) and high negative predictive value (75%). Predictive findings were confirmed through analysis of data from 2 independent trials. CONCLUSIONS: Using a data-driven approach, we identified decision rules using early change in the scores of selected PANSS items to accurately predict longer-term treatment response or non-response to atypical antipsychotic therapy. This could lead to development of a simple quantitative evaluation tool to help guide early treatment decisions. TRIAL REGISTRATION: This is a retrospective, non-intervention study in which pooled results from 6 previously published reports were analyzed; thus, clinical trial registration is not required.

The mean does not mean as much anymore: finding sub-groups for tailored therapeutics.

BACKGROUND: The genomics revolution is still in its infancy, and there is much to learn about how to transform biological knowledge into useful medicines to further public health. At the bedside, we are asking how and why individual patients respond to different drug treatments in different ways. In addition to genetic mechanisms, there are many clinical markers (e.g. medical history, disease severity) as well as social/environmental factors (e.g. smoking habits) that can be used to identify who may or may not respond to treatment. PURPOSE: This issue has some considerable statistical complexity, and different approaches to the analysis of clinical trials may yield more interesting insights into the problem. Novel applications of statistical methods will be discussed, and examples will be used to demonstrate sub-group identification. METHODS: In order to evaluate many potential predictors of response, we use recursive partitioning methods to identify predictor variables and their cut-off values to define sub-groups of patients with differential treatment response. Validation of this variable/model selection approach was done using independent data from other clinical trials. RESULTS: In one example, a classification tree was developed using baseline measures to define important sub-groups of patients that responded much better than the overall mean response in the study. In a second example, a classification tree was built based on measures of response early in treatment to predict longer-term responders and nonresponders. Limitation Classification algorithms can be prone to over-fitting, and validation of results is an important consideration. Obviously, analyses are limited by the available predictor variables. CONCLUSIONS: Using classification trees proved to be very useful in evaluating large numbers of potential predictors to find sub-groups of patients with exceptional response. The method is easy to use, and clinicians can easily interpret and implement results. This approach can be helpful in tailoring treatments to individual patients.

Post hoc analysis: early changes in ADHD-RS items predict longer term response to atomoxetine in pediatric patients.

Data from 5 atomoxetine trials in pediatric outpatients with attention-deficit/hyperactivity disorder (ADHD) were divided into training and validation data sets to develop models predicting atomoxetine treatment response, using changes in individual ADHD Rating Scale (ADHD-RS) items early in treatment. Treatment response was predicted after 1 week by a > or =1-point score decrease in ADHD-RS item 15 ("easily distracted;" positive predictive values [PPVs]: 84.9%, 74.3%, and 73.3%; negative predictive values [NPVs]: 52.6%, 50.5%, and 46.3%; training and 2 validation data sets, respectively); after 2 to 3 weeks, by a > or =1-point score decrease in ADHD-RS item 1 ("fails to give close attention or makes careless mistakes;" PPV = 77.7% and 77.9%) and by the absence of a > or =1-point score decrease on ADHD-RS items 1 and 10 ("on the go;" NPV = 72.2% and 77.5%), or by the combination of items 1 and 10 (PPVs: 75.1% and 75.4%; NPVs: 72.2% and 77.5%; training and validation data sets, respectively).